MASSACHUSETTS INSTITUTE OF TECHNOLOGY 


A.I. LABORATORY 
SEPTEMBER 20, 1977 


A.I. Memo 442 

• »•* 

Towards a Theory of Local and Global in Computation 

Harold Abelson 


Abstract 

We formulate the rudiments of a method for assessing the difficulty of dividing 
a computational problem into independent simpler parts." This work illustrates 
measures of complexity which attempt to capture the distinction between "local 11 
and "global" computational problems. One such measure is the covering multi¬ 
plicity, or average number of partial computations which take account of a given 
piece of data. Another measure reflects the intuitive notion of a "highly inter¬ 
connected" computational problem, for which subsets of the data cannot be 
processed "in isolation." These ideas are applied in the setting of computational 
geometry to show that the connectivity predicate has unbounded covering multi¬ 
plicity and is highly interconnected; and in the setting of numerical computations 
to measure the complexity of evaluating polynomials and solving systems of 
linear equations. 

This is a preprint of a paper to appear in The Journal of Theoretical Computer 
Science. 





Towards a Theory of Local and Global in Computation 

Harold Abelson 


1. Introduction 

* % 

Many approaches to computational complexity focus on issues concerning the speed of 
computation: How. many basic operations are required to compute a given function? How can 
computation time be decreased by performing operations in parallel rather than serially? 
What are the time-space tradeoffs for a given class of algorithms?; There is, however, another 
Kind of complexity. This is the organizational, or structural complexity of processes realized 
by large numbers of interconnected elements. Structural complexity is an important concern 
in many of the difficult areas to which computational methods are just beginning to be applied. 

For example, in exploring computational models for vision [6], one is struck by the fact that 

♦ 

for a contemporary digital computer, the ratio, of connections to components is about three, 
whereas for the mammalian cortex it lies between 10 and 10,000. A comparison such as this 


raises a challenge for theoretical computer science: Is it possible to characterize those 
computational problems uhose solution is inherently better suite* to a highly interconnected 
. structure than to a weakly interconnected structure? 


A similar issue arises in the study of distributed computation and computer networks. 
In this setting, an entity such as a data base might be widely distributed among the nodes of a 


network. Analyses of sorting or searching the data base must be concerned not only with 
elementary operations but also with internodal communication. Consider, for example, solving 
a large system of linear equations, where each column of the matrix resides at a different 
node. If the network is far-flung, the time and cost of communication could dominate the 
solution process, and arithmetic operations performed at individual nodes might be viewed 
simply as overhead. In a case like this, one would be concerned with minimizing, not 
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necessarily the number of operations, but rather the total amount of information which must 
be shipped across the network. What complexity measure is appropriate to this problem? 

Many of the issues in structural complexity revolve around the notion of "local and 
global" or "parts and wholes." Loosely speaking, a computational problem is inherently local if 
it can be divided into small, weakjy interacting modules. A computational problem is inherently 
global if any way of dividing it into pieces must entail substantial interaction among the 
pieces. Creating a useful theory of local and global is of course a formidable task, and this 
paper can be no more than an initial attempt. I introduce a measure, called the covering 

multiplicity t which reflects the organizational complexity of a problem in the sense hinted at 

# ■ 

above. Covering multiplicity is, roughly, the number of independent parts of a process which 

. * ■ _ . -. 

must take account of a given piece of data. In visual processing this might be, for example, 
the average number of "low level" elements influenced by a given patch of the retina. The 

concept of covering multiplicity surely does not. capture all of what might be meant by "local 

* ■ . - • 

and global in computation;” but it is at least a start. 

* - ’ * . * 

This introduction continues with a review of the setting established by Minsky and 

Papert [7] in their analysis of the perception. We will make use of the same basic framework, 
although many of Minsky and Papert’s techniques, relying fundamentally on the linearity of the 
perceptron’s decision element, are unsuitable in the present, more general setting. Section 2 

* * ' * t 

of the paper begins the formal presentation, providing definitions both of covering multiplicity 
and also another complexity measure based on the idea of a "highly interconnected" 

computation. We find that computations which determine whether or not a geometric figure is 

* • 

connected must exhibit arbitrarily high covering multiplicity, and must be highly 

interconnected, thus providing a justification Of Minsky and Papert’s intuitive guess [7]: 

"We chose [in studying perceptrons3 to investigate connectedness because 
of a belief that this predicate is nonlocal in some very deep sense." 

Section 3 turns from geometry to the computation of real-valued functions and gives a 
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necessary and sufficient condition for "computational decomposability" which is used to 
identify multivariate polynomials whose evaluation requires arbitrarily high covering 
multiplicity. We also discuss the matrix problem cited above. The conclusion notes some of 

the many questions which are left untouched in this initial treatment of local and global 
complexity. 

1.1. The Perceptron framework 

Minsky and Papert’s theory begins with an idealized reiino R, which is simply a 
collection of n points. Figures on the retina are subsets X c R. We can think of R as the 
squares in a two-dimensional plane grid and '‘arbitrary geometric figures" as approximated by 
some collection of squares. A predicate on R is a function / from figures on R to {0, J}. The 
support of f is the set of all points of R which affect the value of /, and the order of / is the 
size of support(/). A perceptron is a predicate which has the form 

: nx )»r r. a.f.a) > o i 

v/here f. are predicates and fl, a Jf a 2 > ... are real numbers. (We follow Minsky and Papert in 
using the notation Tsome condition! to signify the predicate whose value is I if the condition is 

true and 0 if the condition is false.) The order of the perceptron is the maximum order of any 

- , . » 

of the Minsky and Papert characterize the complexity of geometric predicates in terms of 
the order of the perceptrons which compute them. They demonstrate, for example, that the 
predicate fX is connected! has infinite order, that is, the order of any perceptron which 
computes, connectivity must become arbitrarily large as the size of the retina becomes large. 
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1.2. Insufficiency of the perceptron analysis as a general theory of local and global 

The notion of order has considerable appeal as a characterization of "local versus 
global": A low order perceptron is local — the partial predicates can make independent 
computations based on small patches of the retina. A high order computation is global — 
individual partial predicates must access portions of the retina. Unfortunately, this 
characterization fails when we consider structures more general than the perceptron. 

A perceptron can be viewed as a composition of functions f « g(f^ f^ .. ^ f r ) where 
g is a predicate on Boolean r-tuples. In a perceptron, g must be a linear threshold function. 
We would like to consider more general computational schemes in which there are no 
restrictions on g . Extending Minsky and Papert’s results to this more general setting, 

i * • * 

* ■ ■ i • . * . *■. . . • 

however, raises many problems. Consider their paradigm result: Connectivity is not of finite 

m i * - . * 

order. This follows from an analysis whose main step is: Parity, that is, the predicate 

\ - ■ _ ■ , ' _ 

\X contains an even number of points! 

is not of finite order* But this can be true only in the linear threshold context. If we omit this 

restriction, then the determination of parity can be as "local" as we please: For any arbitrary 

.... . _* ■ * .* 

division of the retina into disjoint sets S* let 

/j(X) -TX O S- has even parity! 

Then X has even parity if and only if the product over all » of 2/.(X) - I is positive. 

To obtain a hold on what makes the above parity computation "local" and why we 
suspect that any connectivity computation must be "global" notice that for parity the supports 
are disjoint — each point of the retina is examined by only one partial predicate. Moreover, 

any division of the retina into disjoint sets can serve as the partial predicate supports for 

• • . • , * ~ 

computing parity. Let’s examine in this light a predicate which we might agree is "local," the 
property of being locally convex , which can be be determined by checking that X has 
non-negative curvature at each of its boundary points. Even though this determination is 
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"local," it cannot be easily realized with disjoint supports. Suppose, for example, that we 
divide the retina in two disjoint halves and attempt to compute local convexity. Consider the 
shape in Figure 1. Although it is not convex, the point of negative curvature will be 
undetected in either support. But this is hardly a fundamental problem. We merely need to 

allow a bit of overlap as in Figure 2, so that a few points of the retina lie in more than one 

» * 

support, and our local computation can proceed without problem. We suspect, though, that no 
such simple scheme can work for connectivity. In determining connectivity, we would guess, 
points must in general be accessed by many partial predicates. This provides motivation for 
our definition of covering multiplicity: 

The covering multiplicity p is the average number of partial predicates 
tohich examine a given point on the retina. 

• • > • . •* r .’ 

2. Support structures for geometric predicates 
2.1 • Basic definitions 

Definition 2.1.1. For any function of n variables f(xp ..., x n ) the set R ** {i,_, n} is called 

the retina of /. The support of / is the smallest set S C R such that 

i 

f^ x V * • ’’ x n* m R y V • • •’ y n* 

whenever x^ - y^ for all t € $. 

Throughout Section 2, we shall assume that the functions concerned are Boolean functions of 
Boolean variables, although later on we shall also consider real-valued functions of real 
variables. Note that if R is finite, then any function has a unique support. 










Figure 1 

With disjoint supports the point of negative 
curvature is missed 
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Figure 2 

Curvature is detected using overlapping suppotts 
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Definition 2.1.2. A support structure on a retina R is a sequence of the form 

// - (S,. S f ) 

where Sj C R . Note that the Sj need not be distinct. The number r is called the rank of Jf. 
The order of H is the maximum over j of |S For any A cR define co v ^(A) to be the 
number of supports of the structure which intersect Ax 

cov Jf (A) « [{/: A H Sj t 0}\ 

The covering multiplicity of It is the average over R of the number of supports containing a 
given point: 

p(H) - (1/n) cov<{{}) 

« . . „ , ■ . * ’ - * 

* ' ■ • 1 

Note that the sum over R of cov({i}) is precisely the number of pairs (i, Sj) where i e Sj. 
Therefore, if all the Sj have the same size k, then the covering multiplicity, the order, the 
rank and the size of the retina are related by p(H)n ~ kr , 

. « * . 

Definition 2.1.3 A support structure H - ($•} is said to admit a function f(xp . . x n ) if / can 

» 

be represented in the form 

where support(/|) * Sj for r. 

If / is a function from {0, to {0, ]}, i.e., a "predicate on /?," we might try to define 
the covering multiplicity of / to be the minimum covering multiplicity of any support structure 
which admits /. But this will not work, since any predicate is admitted by the multiplicity ./ 
structure ({/}, {2} 9 . . ., {n}>. Covering multiplicity, therefore, is not a useful measure of 
complexity when considering structures consisting of many small supports. We will 
concentrate on the opposite situation, in which we attempt to keep the ranks of the structures 
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bounded for large retinas by using larger and larger supports. One example of this Kind of 

. . » - . ■ 

structure is the fractional support structure, in which each support is some fixed fraction of 

» 

the entire retina: 

Definition 2.1.4. Let M be a positive integer. A support structure if on 8 retina R is. said to be 
a 1/M-fractional support structure if each support in H has size n/M. 

(If M does not divide n evenly we suppose each support to have size within ±1/2 of this 
value.) 

Strictly speaking, of course, a predicate / is defined only for a particular retina, so it 
makes no formal sense to speak of "computing / On large retinas." On the other hand, we can 
think of properties like "parity" and "connectedness" as defining entire families of predicates, 
one predicate for each retina. We can now define the covering multiplicity of such a predicate 
(family) in the context of fractional supports. 

Definition 2.1.5. For any predicate / on fi and positive integer M we define p(/, M f R ), the 
covering multiplicity of f for 1/M-fractional supports to be the minimum p of any 
1/M -fractional support structure which admits /. 
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Definition 2.1.6. A predicate family f is said to have covering multiplicity at mo$t B for all 
fractional supports, pt{f\ frac) < B, if, for all Af, p{f, Af, R ) is uniformly bounded by B on large 

retinas. That is, for any Af there should exist a bound n ^ such that fi(/, Af, R) < B for any 

» , 

retina R with \R\ > If such a finite value B exists, we $ay that / has finite covering 
multiplicity for fractional supports . 

• __ . v * ’ ... 

.* 

2J2. Examples 

To illustrate the above definitions, we compute the covering multiplicity for 

* * 

I/Af -fractional supports for the local convexity predicate, using the the computation outlined 
in Section 1.2. We noted that local convexity can be determined by examining each boundary 

point of the figure in question, and that this can be done with supports that do not overlap, 

. - * * * . 

except for small "interfaces** along the edges. 

o 

Suppose that the retina is a square A units on a side., n - A . Using supports of order 
n/Af we divide R into Af strips, each strip a rectangle of size A by A/Af. We must also assign 
additional partial functions to examine the interfaces between the strips. (See Figure 3.) Each 
interface is itself a strip of width 2, and thus one additional support (of order A^/Af) can 

contain h/2M interfaces (each of size 2h). Since there are Af — I interfaces in all, we need 

♦ 

2M(M-l)/h additional "interface supports." So the entire structure for local convexity has 

r « rank - ^(strips) + *(interface supports) « Af + 2Af(Af-J)/A 

k - order - A^/Af 

p - kr/n - 1 + 2(M-l)/h - l + 2(Af-i)/Vn 

Thus we have shown 
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Figure 3 

Retina divided into strips 
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Example 2.2.1. On a square retina R , predicate / - [X is locally convex! has 

p(/, M, R) < I + 2(M-1)/Vn 

Consequently, for any c > 0, we have p(/; frac) < i + c . 

* ■ ■ ■ * . . * * 

As a second example, consider the predicate IX is a single* solid rectangle!. This can 
be computed by partial functions which "count the corners of X." That is, a figure X is a single 
solid rectangle if and only if its boundary contains precisely four "convex corners" (Figure 4a), 

no "concave corners" (Fig. 4b) and no "double corners" (Fig. 4c). This computation can be 

■ * * - 

performed with almost the same support structure as used above, i.e., by dividing the retina 

. * » ■ _ 
into disjoint strips together with "interface" strips of width 2. This time, however, we assign 

, ' ' • , c ' ■ 

. • * 

three partial functions, hence three supports, to each of the strips. In each support A, the 
three corresponding partial functions /j, /^ output 0 or i as follows: 

L If X n A contains concave corners, double corners, or more than four convex 
corners, then all/. output 1. 

2. Otherwise the three functions output the number (from 0 to 4, counting in binary) of 
convex corners of X D A. 

Using this information, the function g can determine whether or not X is jb rectangle. The 
covering multiplicity here is three times as great as for local convexity: 

Example 2.2.2. If / is the predicate TX is a single sojid rectangle! then for any « > 0, 
pifc frac) < 3 + c . 

These examples illustrate predicates that can be computed with small covering 
multiplicity. On the other hand, we find that no fixed bound on covering multiplicity can 
suffice for computing arbitrary predicates on large retinas: 
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Corner clusters for detecting rectangles 
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Proposition 2.2.3. Let M be a positive integer , and € a positive real number. Then any 
l / M-fractional support structure H which admits all predicates on a retina R must have 

p(H) > (n - e)/M 

provided r is sufficiently large with respect to M. 

« 

n It 

Proof. There are 2 predicates on a retina of size n. Consider, on the other hand, the number 

» ' . • ’ ‘ - 

of predicates admitted by support structures of rank r and order fc. For each of the r partial 

predicates f • there are ^ n ^ ways of choosing support{/«); and having selected a support, 

ak 2 r 

there are then 2* functions /. with that support. In addition, there are there are 2 

possibilities for g. So if the structure admits any function of it variables, we must have 

I 2 ’ s (" 

Using the fact that ( ? ) K an< ^ * a ^' n 8 logarithms, gives 

2 n <rn + f ±r2 k 

or, since k - n/M and r - nfi/k - Mfi 

2 n < 2 M P + Mp(n + 2 nfM ) □ 

2.3. Ifighly interconnected computations 

Covering multiplicity is a measure of the "globalness" of a computation, which can 
provide meaningful results about families of supports in which the order grows large along 
with the size of the retina. For arbitrary supports we can consider instead the question of 
how much "interconnection"-among elements of the retina is required for a given computation. 
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Definition 2.3.1. If H « ( Sj,.. S f ) is a support structure and A c R define Con jj(^) c to 
be the set 


Conjj(/I) - U { S| : S* ft A ? 0 } 



Intuitively, imagine that H is "wired" by connecting together ail pairs of elements in each 
support. Then Con(/l) consists of all those points of R with "direct connections" to points of 
A. By way of analogy with 2.2.3, we will show that any computational scheme which can 
compute arbitrary Boolean functions must be "highly interconnected" in the sense that, for any 
A C /?, either A must intersect many supports or else Con(/t) must contain essentially all of R. 

Definition 2.3.2. A family of support structures H on retinas R is said to be highly 

* / ■ 

interconnected if, given any positive integer B and positive real number €, one has that if n is 
sufficiently large with respect to B t then for any subset A c R with \A\ > «n, either ’ 

cov Jf (A) > B or else |Con^(/l)| > (l-f)n. A predicate / is said to be highly interconnected if 

* * 

any family of support structures which admits f on large retinas must be highly 

* • - „ . 

interconnected. 

, » 

t- 

Proposition 2.3.3. Any family of support structures which admits arbitrary predicates must be 
highly interconnected . More precisely , */ II is a support structure which admits arbitrary 
predicates on a retina R, then , for any A C R , either covjj(A) > |/ 1 | or else 

|Conjj(/l)| > n - log£<covjf(/g)) 

This will follow from another simple counting argument: 
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Lemma 23A, Given collection* of Boolean variables X m (xj ,. . x Q ) and Y » (yj, . . ., y^), let 
t be any integer such that t < a and log^t < b. Then there exist functions f: X X Y {0, 1} 
which cannot be represented in the form 

* 

f{X, Y) ~ g(f t (X ),..., f t (X), Y) 


Proof. Functions f:X x7 *» {0, i} are equivalent to functions f* from X into {0, i}^, the set 
of Boolean functions from Y into [0, 1}, via the correspondence [/*(X)](F) * /(X, Y) 
Representing / in the required form is equivalent to finding a factorization of /*: 




F. 




■{o, iy 


where F is a set of i Boolean variables; $(X) - (/|(X),...,/,(*)) and U*(F)]<F> - g(F,Y). 
To demonstrate that not all functions factor in this way, we need only consider the sizes of 
the sets involved: An element of X can take on 2 a possible values, and there are 2^ possible 

y m 

values for an element of {0, 1} . To construct a function / which does not factor, first 
enumerate the elements of X: 


and the elements of {0, ly : 


v V v 2"'" v 2 a 


% 2 * ■' •' r £ 

r 


There are two cases, depending on the relative sizes of a and b: 

1. If 2^ > a > t let f*(v>) « £. for i • I, —, 2 a . Then the image of f* contains 2 l 

distinct elements, so that f* cannot factor through F, which has size 2 l < 2 a . 

2. If a > 2^ >4 let f*(v>) « for t « 1,.. 2 2 . Then the image of f* contains 2 2 

distinct elements, so that f* cannot factor through F, which has size 2 l < 2^ . □ 








Local and Global in Computation -- September 20, 1977 — 13 


Proof of 2.3.3. For any A c R define R(/l) c R to be the set (R-Con(/9)) U A, i.e., delete from R 
all variables which lie in Con{/l) but not in A. The inclusion of R(A) in R induces a surjection . 
{0, 1}^ -» {0, 1}^^\ Set ( - cov(A), X to be the variables in A, and Y to be the n-|Con(/!)| 
variables in R(A)-A. Then any predicate on R which is admitted by H induces a predicate on 
R(/l) of the form gif^X), f 2 (X) ./,<X), Y). Now apply 2.3.4. □ 

In the case of IfM -fractional supports, the following combinatorial argument shows 

that interconnectedness implies high covering multiplicity: 

. • * - ' 

Lemma 23.5. If If is any family of highly interconnected 1 fM-fractional support structures 
then p{H) £ Af . Consequently, a highly interconnected predicate cannot have finite 

multiplicity for all fractional supports . 

Proof. First note that if A c J? with cov(/l) < Af-1 then |Con(/l)| < n{M-l)/M. So if the family 
of support structures is highly interconnected, we can select any number a and be sure that, 
for R large enough, we have cov(/i) < Af-I implies J/1J < in. 

» 

For any of the retinas R in the family, let R^ be the subset of R consisting of points 
lying in fewer than Af supports: 

R m ~ {ieR: cov({i}) < Af} 

.Note, in particular, that R^ consists of all points with covering multiplicity 0. 

Then cov({i}) + cov({i}) 

Af Af 

But 2 ieR I/ cov U i }>>|R M l-|R ] l 

Af 

and ^ieR-R — Af(n-|K^|) 

nt 

Therefore, we have fttt > Mtt - (Af-1)|R.,I ~ |Rj| 

or M > M - {M-l))R M \/n - flRjl/n) (1) 
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If <r c {Sj} is some collection of the supports in //, define R to be the subset of R 
consisting of those points which are contained in only those supports which lie in <r: 

R ff « { »€ R: i £ Sj for Sj £ <r } 

Consider now the union of the R ff over all sets a of Af-I partial predicates. This union is 

■■ . * t . ..., -~ , _ ~ 

precisely the subset of R consisting of points i for which cov({*}) < M-L That is, 


R M " U |<r|-AM R <r 


( 2 ) 


If r is the rank of H then the number of sets tr containing precisely M^l supports is equal to 

the binomial coefficient (jJ. /)• Notice that for a given <r, cov(R ff ) cannot be greater than 

/ . • 

|cr|. Thus, according to the remark noted at the beginning of the proof, we have IR^.1 < tn on 
large retinas whenever |<r| * Af-1. Equation (2) therefore implies that 

^ (at - i) en 

Combining this with |Rj| < €n and equation (1) yields: 

n Z M - (M-D « (m- i ) - 

Since the order of H is equal to n/M , we have that r - pM. Substituting this into the above 
inequality and using the fact that K 2 ^-gives 

n> M - * l] (3) 

mi 2 

Now, given any & > 0, choose < small enough so that e [ +1 ] < S and consider 

retinas with n large enough so that (3) holds. Suppose, for n this large, we had fi < 

2 

Then 2* hi < 2 M , so tUM-l>2^ M +1] < S, and 

ft + « [(W-I) 2* M + 1] < ft + 6 < M - i + $ < M 
which would contradict (3). Therefore we must have p > M-&. □ 
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2A. Interconnectedness of the Connectivity Predicate 

« 

Theorem 2.4.1. The predicate TX is connected 1 is highly interconnected. 

« * 

The proof arises as a generalization of the simple observation that connectivity cannot be 

admitted by a support structure of rank 2 in which the retina fs partitioned into disjoint 

* 

halves, as can be seen immediately by considering the connectivity of the figures formed by 
the various combinations S* U Tj shown in Figure 5. This construction has been generalized 
by Papert [9] to show that, if we allow only those structures in which no support intersects 

both the left and right halves of the retina, then the rank required for determining 

# ’ 

connectivity must grow arbitrarily large on large retinas: 

* ' ' • 

* 

* 

Proposition 2.4.2. Suppose II is a family of support structures which admits TX is connectedly 

and that no support in H intersects both the left and right halves of the retinru Then f as n 

* • _ • * 

increases , the rank of H must grow at least as rapidly as Vn. . 

■ ♦ ■■ ■ - * 

Proof. Consider the family of figures illustrated in Figure 6, each consisting of a square, with m 

horizontal lines meeting the sides of the square at contact points xj t ..., x m% yj,*.. ., y m . 
Each pair of Boolean m-tuples X « (*j,.. x m ) and Y « (yj,..., y m > gives a figure F{X, Y) ; 
and it is easy to see that one of these figures is connected if and only if each horizontal line 
is connected to the square, either on the left or on the right. In other words, F(X, Y) is 
connected if and only if XVY. (Recall that XVY, "X or Y," means that for each i we have 

*$Vyj .) 

Let fp .. ., / be those partial predicates whose support lies in the left half of the 
retina. Then, for any figure F - F(X, Y), the a-tuple f[if) m (/j(F),..., / a (F)) depends only 














Figure 6 

Connectivity figures for Proposition 2.4.2 
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bn the m-tuple X, i.e., f[(F(X, Y)) - /^(X). We claim, therefore, that a > m. For, if not, then 

there are two distinct m-tuples X j and X^ with /^(Xj) - But then, for any m-tuple- 

* * * 

y, we have F(X|, K) is connected if and only if F(X^, 7) is connected. On the other hand, 
taking Y m ~Xj and noting that F(Xj,~Xj) is connected, shows that we must have X^V^Xj. 
Similarly, taking Y « ~X 2 gives XjV^X^. Therefore Xj - X^, and so a £ m . Finally, observe 
that on a retina of size it,we can choose the number of horizontal strips to be proportional to 
Vn. □ 

« . * . " - - . - 

, * * _ ■ - ♦ _ ■ 

The next step in the proof of 2.4,1 is to extend the above construction so that the 
"contact points" *• and y • can bp distributed throughout the retina: 

* 1 . i 

Lemma 2.4.3. For any integer m there is a constant K(m) such that if H admits connectivity on 

a retina R 9 and A C R with |>5| > KVn, then either cov{A) > m or else |Con(/l)| > R - KVn> 

• . ' * . * . 

■ ■ • . • 

Proof. Let Rj-c R be the subset of R consisting of all points which do not lie within distance 

- 4 

2m of the boundary of R . Choose points Xj, . . x m lying in A such that (1) each x • is 
contained in Rj (2) For i / /, x* and x • do not lie within the same horizontal row of R, and, 
moreover, the horizontal rows containing x* and x . are at least six squares apart. (See Figure 
7.) Notice that we can do this so long as \A\ is greater than |Rj| plus the size of 6m horizontal 
strips, i.e., \A\ > 14Vn. Next, choose points yj ,..., y m in R-Con(A) such that (1) each y> lies 
in R j (2) the horizontal row containing any y. lies at least 6 units from the row containing any 
y k d ** W or an Y x i* We can do this so long as |R-Con(i4)| > ISmVn. To prove the Lemma we 
show that these x’s and y*s can be used as "contacts" in a family of figures equivalent to the 
figures of 2.4.2. This will imply at once that cov(A) > m . 

We construct the figures as follows: Begin by renumbering the x's from top to bottom, 

• ■ * 

_ ___- - -- a aaaw — —— 




Figure 7 


Choosing the points in Lemma 2.4.3 
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j.e., so that x^ is above x. for *<;. Next arrange each r. to be a pontact point for three 

* 

horizontal "wires'* as shown in Figure 8. Extend each of these wires on the left to meet the 
boundary of Rj. Now do the same thing for the y’s, only this time working to the right. Next, 
in the boundary of Rj, connect the bottom wire of each to the top wire of and 

similarly for the y*s. Connect the top wires for and yj, and the bottom wires for x m and 

♦ 

y m Finalfy, make a connection "around the bottom of R" to join the middle wire of ar. to the 
middle wire of the corresponding y., as follows: for (*j* yj) work in the boundary of R; for 
in the strip 3 units in from the boundary; and so on. These latter connections all lie 
in R-Rj, and so do not interfere with the previous wires. (The final figure is illustrated in 

Figure 9 for the case m - 3.) As in 2.4.2, these are connected if and only if XVY . □ 

. 1 * ■ 

Proof of 2.4.1. Given M and <, let K be the constant K(Af+i) given by Lemma 2.4.3, and choose 
R large enough so that KVn < €n. Then, if |/l| > tn and |Con(/l)| < (I-€)h , we have that \A\ and 
|R-Con(/J)| are both greater than KVn. Hence cov(i4) > M+I by 2.4.3, which shows that the 
computation is highly interconnected. □ 

2.5 Covering multiplicity of the connectivity predicate 

Applying Lemma 2.3.5 to Theorem 2.4.1, we see immediately that connectivity, unlike 

the "local" predicates for local convexity and [X is a single solid rectangle! discussed in 

\ * 

Section 2.2, does not have finite covering multiplicity over all fractional supports. In 
particular, i/Af-fractional support schemes which compute connectivity on large retinas must 
have liw^eof* > M . But connectivity, we expect, should be "even more global" than that — 
more like computing arbitrary Boolean functions, where, for any fixed M , we have 
!* m rt -»eoP - «o by 2.2.3. Indeed, the same reasoning as in 2.4.2 shows that this must be true in 
connectivity computations, and that p must grow as rapidly as Vn, so long as we assume that 






Figure 8 

Details of the contact x 


in Lemma 2.4.3 
















Figure 9 


Connectivity figure for Lemma 2.4.3 





Local and Global in Computation — September 20, 1977 -- 18 


the supports of the partial predicates are disjoint. In this section we show that connectivity 
has unbounded covering multiplicity for J/Af-fradional supports, even if the supports are 
allowed to overlap. We shall not, however, consider the case of arbitrary overlap. Rather, we 
restrict attention to schemes in which the supports overlap "regularly" according to the 

following prescription: 

• • 

’ ' ' '. • . . • • i .. • 

Definition 2.5.1. Let R be a square retina which is partitioned into D square blocks of equal 
size. Then a d/D-regular support structure is a structure in which each support consists of 
some fixed number d<D of these blocks. 

Note that this is a special case of d/D-fractional supports. Supports may overlap, but they 

* * . 

must overlap "regularly" — two intersecting supports must necessarily have an entire block in 

' common. We remark also that, although we phrase our results here only for square retinas, 

- • - # , ........ 

the same sort of things will be true for all families of retinas with a sufficiently large 
"interior," i.e., for sequences of retinas f? in which the perimeter of R grows no faster than 
V|R|. . ; 

Theorem 2.5.2. For any family of dfD-regular support structures which computes connectivity , 
the covering multiplicity must satisfy lim^^fi ■ 

* 

■ . • _ _ . 

The proof uses the switching network construction to translate the problem of determining 
connectivity into that of computing an arbitrary Boolean function. Recall [7] that if 
. . ., x a ) is a set of Boolean variables, and /; X {0,1} is a predicate, then a switching 
network for f is given by a function F which associates to each Boolean o-tuple a figure on a 
retina R such that F(*j, . . ., x a ) is connected if and only if f(xp . . ., x Q ) - i. We can 
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construct such a network by writing /.in conjunctive normal form and translating the Boolean 

expression into a network, interpreting conjunction as series coupling and disjunction as 

’ ■ • 

parallel coupling. For example, the predicate 

f{xp x 2 t - Tat least 2 of the variables are equal to 11 

has conjunctive normal form 

(xjVx^Vx^) A (~Xj Vx^Vx^) A (xjV^x^Vxj) A (tfjVx^V^Xj) 

* 

and the corresponding network is shown in Figure 10. The figure is interpreted as follows: 

• * 

when the variable x* is equal to 1, the squares marked x • are filled in and the squares 
marked "~xj" are left empty. Conversely, when x i - 0, the squares marked "~x." are filled in 
and the squares marked "x." are left empty. These squares are the called the contacts of the 
network. 

Notice that a function of a variables can have at most a2 a terms in its conjunctive 
normal form, and hence a network with this many contacts can realize any Boolean function of 

a variables. This is the sense in which computations for determining connectivity must also be 

*• . . * ' 

able to compute arbitrary Boolean functions. The proof of 2.5.2 now rests on the following: 

Lemma 2.5.3. Suppose tee have D sets of Boolean variables 

*i-<*ll>-xj m >>—> Xj) m ( X D1> • •-> x Dm) ' 

Consider support structures of order dm in which each support consists of some collection of 

d < D of the X», Then for any fixed values of d and D, and any hound B, we can choose m 

large enough so that there are Boolean functions of the X 9 * which cannot be computed by 

structures of this type having covering multiplicity p < B. 

, * ’ - 

This lemma follows at once from 2.2.3. Moreover, by examining the proof of 2.2.3, we see that 
fi would have to grow large with the the same order asm, □ 









Switching network for 



i- • 






















«<?;WV?kWf5!.S 


























Local and Global in Computation — September 20, 1977 — 20 


Proof of 2.5.2. Construct in R a switching network that can realize any function of Dm 
variables, such that the contacts corresponding to the variables in the ith set X* all lie within 
the ith block. Then any regular support structure which admits connectivity on R must also 
admit arbitrary Boolean functions as in 2.5.3. As R becomes large we can choose larger and 
larger values for m. Therefore must also increase without bound. □ 

Remarks. The above proof shows that p must grow at least as rapidly as log n. This 

logarithmic factor arises from the use of the general switching network to realize arbitrary 

.. • ■ • . * • 

Boolean functions. It is natural to ask if there is a bound more in keeping with the Vn growth 
observed for disjoint supports. Also, is there some way to eliminate the "regularity’* 
assumption, and so establish 2.5.2 for arbitrary I/Jf-fractional supports? What is the order of 
growth in this case? 

* 

3. Local and global in real-valued computations 

^Fhe techniques presented above for analyzing Boolean functions are also applicable to 
the study of real-valued functions. The "retina" in this setting is an index set for n real 
variables (x^,. .., x n ), and the support of a function /: JR n -t IR consists of all indices » such 

that / depends on x.. This extension to the real-valued domain is analogous to Uesaka’s 

« .■ 

extension of Minsky and Papert’s work to the theory of "analog perceptrons" [10], [11]. The 
analog perceptron formulation, however, deals only with functions of the form 
fixp . . ., x n ) « 2^ where each /. is a function of (hopefully) fewer than n of the x 7 s. In 
keeping with the comments in Section 1.2, we find that this linearity requirement is too 
restrictive to serve as a basis for a general study of structural complexity. For example, the 
function 
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mult: (xj,. ,r n )Xj * * * 

cannot be written as a sum of functions /V of .'fewer than n of the x’s [10] and is therefore 

i 

"global" from the perceptron point of view; but allowing multiplication, rather than merely 
summation, at the "output stage" would enable us to compute mutt using any partition of the 
(x.,. . ., x n ) as supports for the /.. (This is exactly analogous to the comments on the parity 
predicate in Section 1.2.) 

In the sections below we shall work in the category of real-valued differentiable 

functions. From this point of view we analyze, in Section 3.2, the covering multiplicity 

* 

necessary compute arbitrary polynomials. Section 3,3 applies the same ideas to computing the 
determinant of a matrix and solving systems of linear equations. All of these results are based 

on a theorem on functional decomposition proved in Section 3.1. 

, . ♦ 

' ■ 

3.1. The Decomposition Theorem 

Following the framework of Section 2 we will say that a support structure (Sj,. . ., S r ) 

" _ ' . • 

admits diffcrentiably a function /: R n ■-*> R if f can be represented in the form 

f * #(//> • • *9 f r ) where support(/.) c S^ and g, /j,..., / are differentiable. We will also 

• * 

consider situations in which f is defined only locally in some neighborhood U c (R n in which 
case we require that g and the /. be locally defined. To investigate conditions under which 
this can be done, consider first a simple kind of support structure in which the variables x. 

are partitioned into two disjoint sets X - (xj,..., x fl ) and ^ .*„>• Let Diff(X) 

denote the algebra of real-valued differentiable functions of the variables in X. For any 
/: IR n IR let A(X,/) denote the module generated over Diff(X) by the o functions 
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3.1.1. Decomposition Theorem. // /: V C R° X R n ~ a It is a differentiable function which can 


he represented as 


/<X, Y) - g(fp 0,.. f r (X) r Y) 


where g : R and fg R a R, i ■ i,.. , r are differcntiable f then the module A(X, f) 

has rank at most r throughout U. Conver$ely f if f is continuously differentiable , and A(X, f) 
has rank at most r, then there is an open subset V C U on which f can be so represented . 


Proof. Differentiating the equation / « g(fp ..., f^ Y) with respect to any x> e X gives 

bf/bx.- Xj (bg/bfj) (bfjfbx.) 

Each fp and therefore each bfjfbx j, lies in Diff(X). Hence bffbx• lies in the module 
generated over Diff(X) by the r functions bgfbxj . Since this is true for all i, we have that all 
of A(X, f) is contained within this module and is therefore of rank at most r. 

To show that the condition is also sufficient, begin by choosing any p - (pj£$ Py) € If 
and r+I sets of particular values F j,..,, F r+ j for the variables F such that the pairs (pj^, py) 
all lie in U, and consider the functions /^(X) - /TX, Y»). We claim that for any values F. the 


matrix 


||d/j/d*jll { . j . r+J . j m l . 


has rank at most r. 


To prove this, pick any r+i columns, say, for notational simplicity, columns I through 
r+i. According to the hypothesis on A(X, f) we have 

bHX, Y)/bx r ^ - X jmJ ^ r v r+IJ (X) mx, Y)/bxj) 
where the ^ r+ ^y(X) are functions of X alone. Thus we can substitute any F^ for F in the 


above equation to get 


W bx r+l " X j *r+l,j< X) 


Since the <p r +j AX) are independent of i this shows that 
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((r+l)st column of matrix) m X f . AX) (j th column of matrix) 
which proves the claim. 

Now to complete the proof of the theorem, choose T I . y r + l to maximize the rank 

of ||d/j/d'*y||. Denoting this maximum rank by m, m < r by the claim. Without loss of 
generality we may assume that the first m rows of the matrix are linearly independent. 
Consider now the function 

F-.U-* R m+i x R""° 

where 

F(X, Y) - </«, A /j(X ),..f m (X),Y) 

By choice of the F j, ..F f+ j we have that the rank of F is at most m+n-a. Therefore the 

- • • s » ■ . 

functions describing the image of F are functionally dependent [13- (This is where one needs 
that / is continuously differentiable.) Moreover, the set of points at which the • • •> f m 

, Py) € V we can 

apply the Implicit Function Theorem to solve the functional dependence relation for f(X, Y), 

i.e., there exists a continuously differentiable function g defined in a neighborhood of pj^ such 
that 


have maximal rank forms an open set V c U, and at any such point p * (pj^ 


j\X,Y) - gifjiX), .. f m iX), Y) O 

* 

■ . * * . 

Note: In the case r«i this theorem reduces to a result of Leontief [4} 


To relate 3.1.1 to the framework of Section 2, we define Drff(fl) for any B c [J.n} 

to be the algebra of differentiable functions in all the variables (*■)• » D . 

I I € l) 




Local and Global in Computation — September 20, 1977 — 24 


Corollary 3.1.2. Suppose II is a support structure which admits differentiably the function 
f: IR n -► BL and A C {1,. . ., it}. Then among the a derivatives (bffbx£> € ^ at most C0Vfj(A) 
are linearly independent over Diff(Co r\^(A)), 

Proof. Proceed just as above, noting that, for any i e A 

bf/bx. - X. ( bgfbf.) (bfj/bx.) 

where the summation ranges over j € Cov({i}); and that for any such j 

bfj/bx. € Diff(Con({i})) c Diff(Con(/l)). □ 

3J2. Polynomial evaluation 

We now apply the results of 3.1 to prove that support structures which admit 
arbitrary polynomials must be highly interconnected, and of infinite covering multiplicity for 
fractional supports: 

. _ ' » ■ 

Proposition 3.2.1. // H is a support structure which admits differentiably the polynomial 
P(x„ . . x ) m X: 4 x fxJ then for any A C {1,. .n}, either cov(A) > \A\ or else Con(A) is 

the entire index set {!,. . ., n}. 

■* • 

Proof. Suppose that there is some index i not contained in Con(^). By renumbering the x’s we 

may assume that this index is n, and that A « {1,_, |/i|}. (This renumbering does not change 

P, which is symmetric in the x..) Then Diff{Con(/0) c Diff(xj,..., x n _j). Moreover 

P(x .... x ) * 2* * x .xJ * X • x .X J + J., 2 . x *x»7 
\ J* 9 n' i,J J i J j n Mn j i 

m x n + 2 x .x J + 2-/ x x» n modulo Diff(x.. #) 

n ji*n j n itn n i ' P 

and so, taking the derivative with respect to x^ (Wn) gives 

bP/bXfc » x n k + modulo Diff(xj,.. ., x n _j) 
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As the reader can verify, this equation implies that the module generated by (dP/da^)^^ 
contains the \A\ elements x n , x n 2 f . .. which are linearly independent over 

Diff(*j,.. ., x n _|) and hence over Diff(Con(/I)). Therefore, by 3.L2 we have co v{A) > \A\ . □ . 

Corollary 3.2.2. Any family of support structures which differentiably admits arbitrary 
polynomials must be highly interconnected. Consequently in any family of such structures 
with 1 /M-fractional supports the covering multiplicity satisfies lim^^ p > M . 

* ■ • ■ * * , 

This follows at once from 3.2.1 and 2.3.5. □ 

33 Matrices and linear equations 

We turn next from the computation of arbitrary polynomials to consider the specific 

polynomial which expresses the determinant of a matrix in terms of the matrix elements. The 

retina here is an index set of real variables which we view as a matrix 

II nn 

X m ll*jyll’ The function det: (sr-y) det(X) is a polynomial of degree n. Expanding det by the 
ith row shows that 

(bfbx Si ) det - ± X ij ' ' 

where is the cofactor of x*j t itself a determinant of order n-J. 

. . . • 

Lemma 3.3.1. Let S m (sp . .., *■ ) be any collection of n of the variables x* .. (That is 9 each 

•y ^ 

represents some Let X^ be the co factor of sThen the co factors XX n are 

linearly independent over Diff (sp . .., * n ). 

Proof. By permuting the rows and columns of X and reordering the s^s we may assume that 

i 

*1 mx lV "^ e c °f ac f° rs X ,..X K are sums of monomials of degree n-1. The key to the 
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Lemma is the claim that, in the expansion of X 11 there is at least one monomial which does not 
contain any of the ar’s lying in S. 

To prove this claim we note that, since X^ is itself a determinant of order n-1, the 
monomials at in the expansion for X^ are the products oc m ot» . . . ct„ where at - lies in the 
/th column of X and no two cij$ lie in the same row. So we must show that there is at least 
one such set of cty’s, none of which lie in S. Let c - be the number of elements of S lying in 
the /th column of X. By permuting the columns, we may assume that c»>c«> *.. >c . There 

a— ft 

are two cases to consider: 

Case I: Cj > 1. We want to choose a 2 from the second column. There are n-f-c, possible 
choices. Likewise, in choosing ttj from the third column, and not lying in the same row as 

a 2> there are n-2- cj choices. In general, there are choices for We need to 

know then, that 

n> Cj + (j~l) j m 2,n ( 4 ) 

• " * * * 

in order to be sure that there are choices possible at each step. But 2c . - n and 

e l— c 2- ' • • — c n s0 c j ^ "//• A ,s0 > if c j > 1 we can be sure that c. - 0 for j > tt/2. So 
we need only verify (4) for j? < n: 

Cj + U-I) < (n/j) + (j-1) - [0'-n)(/-I)//]Vn < n 
Therefore we can construct a as required. 

Case 2: cj - i. In this case each column of X contains one element of S. Now, if some row 

contains more than one element of S we can prove the result by applying Case 1 to the 

transpose of X (which has the same determinant as X). Otherwise, if each row and column 

contains one element of S, we may permute rows and columns so that the are precisely the 

diagonal elements of X. If so we can take the monomial a to be, e.g., X« 9 X»/**-* X ,X « . 

° 23 34 n 9 n-l n-l 9 r 

This proves the claim. 

Now to complete the proof of the lemma, let F « Diff^,..., $ n ) t and suppose that the 
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L 

X were linearly dependent over F, i.e., 

" 0 where f k e F 

Now let o be the monomial in X 1 whose existence is assured by the above claim, and set 
m a+(3. We woujd then have 

!,« - -Ifi - „ 4*‘ (S) 

U1 . . n !_ n be ,hose *ij whi ' h do not lie In S and regard (5) as an equation in the 

polynomial ring F[t p . . t 2 } The construction of a guarantees that, as an element of this 

n ~n 

ring, o has degree n-i. Also, the terms on the right side of (5) have degree at most n-L 
Moreover, none of these terms can be equal to a, since a monomial in the expansion of the 
cofactor of an element of a matrix cannot also lie in the expansion of a cofactor of a different 

element of the matrix. Therefore equation (5) is impossible. □ 

i • " ' 

• . 

As a consequence we deduce a theorem about the distributed computation of 

determinants: Suppose we have an nxn matrix X, whose entries we partition into n sets S- of 

order n. (The S- can be the rows, columns, VnxVn submatrices or whatever.) Consider 

computing det(X) by first evaluating separate "preprocessing" functions /. of the and then' 

combining the results of the f. by some function g. We find that computing a determinant is a 

* * . * 
worst case for this kind of distributed computation: 


Theorem 3.3.2. Let If ~ (S^,..., S n ) be a support structure for functions of n 2 variables where 
the Sj are pairwise disjoint ani of size n. If H differentiably admits the determinant 

function, then cov(S ) > n for all j. 


Proof. Since the Sj are disjoint, we have Con(S y ) *» Sj and the theorem follows immediately 

from 3.3.1 and 3.1.2. □ 
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In other words, no matter what "local" computations we make based on the n elements of each 
Sj f we still must transmit n numbers to be combined by g. From the communication point of 
view, there is no point in doing any local computations at all. We may as well transmit the h 
elements of S• to g directly, 

As a final application of these ideas, consider the problem of solving an nxrt system of 
linear equations Xy * b based on dividing the matrix X into columns. That is to say, each of 

the elements y • should be computed by a structure in which each support can intersect at 

* * 

most one column of X (and well also allow each partial function to access any entry of 6). As 
with determinants, this turns out to be a worst case: the information needed from each 
column of X cannot be transmitted by fewer than it differentiable functions: 

Theorem 3.3.3. Let b be a fixed non-zero n-vector 9 and let yj(X) be the j-th entry of the 
solution to the system of linear equations Xy ** 6. Suppose that H is a support structure 
tohich diffcrentiably admits y and that each support in II is contained within a single 
column of X. Then p for the j-th column X • of X, we have cov(X -) > n. 

■ ■ ■ : • • . • _ • 

» " * 

Proof. Let 6-*X denote the "augmented matrix" in which the j\ h column of X is replaced by 6. 
By Cramer’s Rule, y • «* |6^X|/|X|. Hence, for any element x 

*yj/** ik - I/m 2 [ (\xmbix\/d Xik ) - (\bixmx\/bx ik ) ] 

For any /, if b is non-zero, there are matrices. X for which |X| if 0 and (fc^X) r* 0. So consider 
the elements in the j\ h column of X, i.e., take j « k in the above equation. Since |6^ # X| does 
not involve the variable x»j 9 b\b^ X\/bx*j « 0 and so 

brj/^ij - - (\b j 'x\/\x\ 2 ) bm/bx^ -± (\b>xwfmi 

Bijt, by Lemma 3.3.1 the X 2 j, .. X n / are linearly independent over DifHxjj, .. ., * n j). 
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♦ 1 % 

Hence the multiples of these polynomials by \b*X\j\X\* are also linearly independent. 
Corollary 3.1.2 therefore implies that cov(Xy) > ru □ 

» * 

* 

3A Relations with Hilbert's I3th problem 

The preceding perspective on the complexity of functions is related to investigations 
growing out of Hilbert’s 13th problem, which concerns the possibility of representing functions 
of several variables as superpositions of functions of a smaller number of variables. (For a 
survey see [5j) Notable among these is the result due to Kolmogorov and ArnoPd [3] that 
any continuous function can always be expressed as a superpostion of continuous functions of 

two variables. On the other hand, it is known that this cannot be done if the functions in the 

♦ • 

decomposition are required to satisfy differentiability constraints. This qualitative difference 
between differentiable and non-differentiable decompositions suggests there is no 
straightforward extension of the techniques of the previous sections to allow for 
non-differentiable partial functions. 
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4. Conclusion; Questions for further research 

This paper has suggested precise computational formulations which interpret such 
vague notions as "global," "gestalt" or "the difficulty of dividing a computation into 
independent simpler parts." The goal is to develop meaningful measures of complexity which 
reflect only how the pieces of a computation are interrelated, and are independent of the 
specific operations performed by each piece. Hopefully, such an effort could lead to a unified 
perspective for discussing problems of "parts and wholes" in computational geometry, in 
numerical computation, in distributed data processing, and perhaps even in artificial 
intelligence and cognitive theory as indicated in [8J. 

The theory is still at an embryonic stage, and the reader will no doubt recognize 
numerous ways in which the above results can be improved. What can one say about the 

order of growth of covering multiplicity for connectivity or other geometric predicates? Are 

• ' * 

there general techniques for establishing lower bounds for the covering multiplicity of specific 
Boolean functions? The Decomposition Theorem of Section 3.1 is a differentiable analogue of 

results of Ashenhurst on "disjoint decompositions" in switching theory [2]. Can this be 

* • - 

extended to more general decompositions? We have only hinted at applications to the study 
of distributed data bases. Developing covering multiplicity criteria in this context is surely a 
major area left untouched by the present investigation. Other extensions of the theory could 

deal with continuous retinas," in which the subsets S c R become measures defined on the 

♦. 

plane, and the predicates f become functions on the Hilbert space of measures. It is also 
important to develop alternative measures of "local and global" complexity and contrast these 
with covering multiplicity. For example, there should be a whole spectrum of. 
interconnectedness" running from non-overlapping support structures to the highly 
interconnected structures defined in Section 2.3, and one should be able to measure precisely 
the "intrinsic interconnectedness" required for computing a function. Certainly, much remains 
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* * . ‘ . ■ 

to be done in this area. 
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for the manuscript. 
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